Supporting reference-based communication in instant messaging systems

ABSTRACT

Modifying captured portions of references in an instant messaging system is supported. An electronic message that includes a captured portion of a reference is received. The captured portion is a portion of the reference that is selected and inserted into the electronic message by a sender. The captured portion is presented within the electronic message on a user interface. A query for data that is associated with the reference is sent to a database that stores a supplemental object, and in response, information extracted from the supplemental object is received. The visual representation of the captured portion is modified within the electronic message based on (i) the information extracted from the supplemental object and (ii) an interaction with the captured portion that generated the query, and such that more than the captured portion of the reference is presented within the electronic message without an additional action from the sender.

TECHNICAL FIELD

The present invention relates generally to the field of electronicmessaging and, more particularly, to supporting reference-basedcommunication in instant messaging systems.

BACKGROUND

Instant messaging (IM) is a type of communication that offers real-timetext transmission over a network, such as the internet. Short messagesincluding one or both of text and images are typically transmittedbi-directionally between two parties. Some IM applications can use pushtechnology to provide real-time text, which transmits messages characterby character, as they are composed. Instant messaging systems tend tofacilitate connections between specified known users (often using acontact list also known as a “buddy list” or “friend list”). Dependingon the IM protocol, the technical architecture can be peer-to-peer(i.e., direct point-to-point transmission) or client-server (i.e., wherea central server retransmits messages from the sender to thecommunication device). More advanced instant messaging can add filetransfer, clickable hyperlinks, Voice over Internet Protocol (VoIP), orvideo chat functionality.

SUMMARY

According to one embodiment of the present invention, a method forsupporting reference-based communication in instant messaging systems isprovided. The method includes: receiving, by one or more computerprocessors, an electronic message that includes a captured portion of areference; presenting by one or more computer processors, the capturedportion of the reference on a user interface; sending to a database, byone or more computer processors, a query for data that is associatedwith the reference, and in response, receiving, by one or more computerprocessors, extracted information based on the query; and on the userinterface, modifying, by one or more computer processors, the electronicmessage based on the extracted information and an interaction with thecaptured portion of the reference.

According to another embodiment of the present invention, a computerprogram product for supporting reference-based communication in instantmessaging systems is provided. The computer program product comprises acomputer readable storage medium and program instructions stored on thecomputer readable storage medium. The program instructions include:program instructions to receive an electronic message that includes acaptured portion of a reference; program instructions to present thecaptured portion of the reference on a user interface; programinstructions to send, to a database, a query for data that is associatedwith the reference, and in response, receiving, by one or more computerprocessors, extracted information based on the query; and programinstructions to, on the user interface, modify the electronic messagebased on the extracted information and an interaction with the capturedportion of the reference.

According to another embodiment of the present invention, a computersystem for supporting reference-based communication in instant messagingsystems is provided. The computer system includes one or more computerprocessors, one or more computer readable storage media, and programinstructions stored on the computer readable storage media for executionby at least one of the one or more processors. The program instructionsinclude: program instructions to receive an electronic message thatincludes a captured portion of a reference; program instructions topresent the captured portion of the reference on a user interface;program instructions to send, to a database, a query for data that isassociated with the reference, and in response, receiving, by one ormore computer processors, extracted information based on the query; andprogram instructions to, on the user interface, modify the electronicmessage based on the extracted information and an interaction with thecaptured portion of the reference.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating a computingenvironment for supporting reference-based communication in instantmessaging systems, in accordance with an embodiment of the presentinvention.

FIG. 2A is a flowchart depicting operations on a source computing devicefor supporting reference-based communication in instant messagingsystems, on a computing device within the computing environment of FIG.1, in accordance with an embodiment of the present invention.

FIG. 2B is a flowchart depicting operations on a target computing devicefor supporting reference-based communication in instant messagingsystems, on a computing device within the computing environment of FIG.1, in accordance with an embodiment of the present invention.

FIGS. 3A and 3B depict an embodiment of a user interface that enables auser of a target computer device to interact with a dialogue that ispresented in an electronic message, in accordance with an embodiment ofthe present invention.

FIGS. 4A and 4B depict an embodiment of a user interface that enables auser of a target computer device to expand a captured portion of areference that is presented in an electronic message, in accordance withan embodiment of the present invention.

FIG. 5 is a block diagram of components of a computing device executingoperations for supporting reference-based communication in instantmessaging systems, in accordance with an embodiment of the presentinvention.

DETAILED DESCRIPTION

Instant messaging is a useful technique for having discussions,requesting information, and passing on information. Embodiments of thepresent invention, however, recognize that a message received over aninstant messaging system may not include enough information for therecipient to respond appropriately to the message (e.g., answer aquestion or forward a desired piece of information) without additionalinformation from the sender. If the sender in unable to respond quickly,a loss of productivity, a delay in decision making, and/or an incorrectdecision may occur. For example, the aforementioned situations may occurwhen a user goes offline after initiating a communication, when a userprovides insufficient information, or when a user unable to makedecision due to a lack of access to information. Embodiments of thepresent invention provide a capability for a recipient of an instantmessage to view information that is associated with a captured portionof a reference that is included in the instant message in response todetecting a request for additional information from the recipient.

FIG. 1 is a functional block diagram illustrating a computingenvironment, in accordance with an embodiment of the present invention.For example, FIG. 1 is a functional block diagram illustrating computingenvironment 100. Computing environment 100 includes instant messagingdevice 105 (IM device 105), instant messaging device 110 (IM device110), and instant messaging server 125 (IM server 125) connected overnetwork 120. IM devices 105 and 110 execute instant messaging software115 (IM Software 115). IM server 125 executes IM authentication software130 and includes reference database 135.

In various embodiments, each of IM device 105 and IM device 110 is acomputing device that can be a standalone device, a server, a laptopcomputer, a tablet computer, a netbook computer, a personal computer(PC), or a desktop computer. In another embodiment, each of IM device105 and IM device 110 represents a computing system utilizing clusteredcomputers and components to act as a single pool of seamless resources.In general, each of IM device 105 and IM device 110 can be any computingdevice or a combination of devices with access to one another, and withaccess to and/or capable of executing IM software 115. In embodimentslike the one depicted in FIG. 1, IM devices 105 and 110 are clientdevices that are communicatively connected to IM server 125 via network120. Each of IM device 105 and IM device 110 may include internal andexternal hardware components, as depicted and described in furtherdetail with respect to FIG. 5. In some embodiments, IM device 105 and IMdevice 110 are the same type of computing device. In other embodiments,IM device 105 and IM device 110 are different types of computingdevices.

In the embodiment depicted in FIG. 1, respective instances of IMsoftware 115 are stored on IM device 105 and IM device 110. In otherembodiments, one or both instances of IM software 115 may reside onanother computing device, provided that each can access and isaccessible by IM devices 105 and 110 respectively. In yet otherembodiments, one or both instances of IM software 115 may be storedexternally and accessed through a communication network, such as network120. Network 120 can be, for example, a local area network (LAN), a widearea network (WAN) such as the Internet, or a combination of the two,and may include wired, wireless, fiber optic or any other connectionknown in the art. In general, network 120 can be any combination ofconnections and protocols that, in various embodiments, will supportcommunications between IM devices 105 and 110 and IM server 125, inaccordance with a desired embodiment of the present invention.

In various embodiments, IM software 115 provides a user interface, IMuser interface 117, which executes locally on IM devices 105 and 110. IMuser interface 117 further operates to receive user input from a user ofa respective IM device via the provided user interface, thereby enablingthe user to interact with the respective IM device via IM software 115.Additionally, IM user interface 117 provides a user interface thatenables users of IM devices 105 and 110 to interact with each other vianetwork 120. In various examples, the users interacts with IM software115 in order to exchange one or more message that are associated with areference, as described herein. Examples of IM user interface arediscussed with respect to FIGS. 3A-4B.

IM software 115 operates to send or receive electronic messages thatinclude a portion of a reference, as described in greater detail withrespect to FIGS. 2A and 2B. As used herein, a “reference” is any dataobject that is associated with and is presented, at least in part, in anelectronic message (e.g., an IM or email). As used herein, a “sourcecomputing device” is a computing device that sends a message thatincludes a captured portion of a reference. Similarly, a “targetcomputing device” is a computing device that receives the message thatincludes the captured portion of the reference. As depicted in FIG.3A-4B, various embodiments of IM software 115 enable a user of a sourcecomputing device to “capture” a portion of a reference. In someembodiments, “capturing” describes taking a snapshot of a reference(e.g., taking a screenshot that includes a dialogue window, in which thedialogue is the reference, as depicted in FIGS. 3A and 3B) and insertingthe snapshot into an electronic message. In other embodiments,“capturing” describes copying and pasting a portion of a file into anelectronic message (e.g., pasting a portion of a text file, as anobject, into an IM, in which the text file is the reference, as depictedin FIGS. 4A and 4B). In various embodiments, a user of a targetcomputing device can use IM software 115, in cooperation with at leastone of IM server 125 and an instance of IM software 115 executing on thesource computing device to obtain more information from the referencethan is available through the captured portion.

Persons of ordinary skill in the art will understand that any onecomputing device (e.g., IM device 110) may be both a target computingdevice and source computing device over the course of a conversationconducted via various IM protocols, email protocols, or other types ofelectronic communication. Accordingly, an instance of IM software 115can, with respect to a first message, provide the functionalitydescribed with respect to an instance of IM software 115 executing on asource computing device and, with respect to a second message, providethe functionality described with respect to an instance of IM software115 executing on a target computing device.

In the embodiment depicted in FIG. 1, IM Server 125 is a computingdevice that can be a standalone device, a server, a laptop computer, atablet computer, a netbook computer, a personal computer (PC), or adesktop computer. In another embodiment, IM server 125 represents acomputing system utilizing clustered computers and components to act asa single pool of seamless resources. In general, IM Server 115 can beany computing device or a combination of devices with access to IMdevice 105 and IM device 110, and with access to and/or capable ofexecuting IM authentication software 130. In various embodiments, IMserver 125 includes reference database 135. IM server 125 may includeinternal and external hardware components, as depicted and described infurther detail with respect to FIG. 5.

In various embodiments, IM authentication software 130 is stored on IMserver 125. In other embodiments, IM authentication software 130 canreside on another computing device, provided that IM authenticationsoftware 130 and IM server 125 can access and are accessible by eachother, and provided that IM devices 105 and 110 can access IMauthentication software 130 while executing IM software 115. In yetother embodiments, IM authentication software 130 can be storedexternally and accessed through a communication network, such as network120.

In the embodiment depicted in FIG. 1, IM authentication software 130operates to receive data that, at least in part, describes a referencethat is associated with a message sent from a source computing device(e.g., IM device 105). IM authentication software 130 stores thereceived data in reference database 135. In various embodiments, thereceived data can include one or both of data and metadata that describeone or more requirements for accessing a reference stored in referencedatabase 135. For example, IM authentication software 130 can receivedata describing various authorized subjects (e.g., a user of a targetcomputing device, such as IM device 110) and/or various rules forimplementing an access control system, such as a mandatory accesscontrol system, a role-based access control system, a discretionaryaccess control system, or another type of access control system known inthe art. Upon receiving credentials that describe a subject from atarget computing device (e.g., credentials of a user of IM device 110)and a query for data describing, at least in part, a reference stored inreference database 135, IM authentication software 130 authenticates thecredentials and determines whether or not the subject is authorized toreceive the data. If the subject is authorized to receive the data, IMauthentication software 130 transmits the data to the target computingdevice.

Reference database 135 is a data repository that may be written to andread by one or both of IM authentication software 130 and variousinstances of IM software 115. Data describing one or more references anddata associated with one or more access control policies can be storedto reference database 135. In some embodiments, reference database 135can be written to and read by programs and entities outside of computingenvironment 100 in order to populate the repository with data describingvarious authorized subject(s) and/or access control rules.

Persons of ordinary skill in the art will understand that instantmessaging services and various other forms of electronic messaging canutilize peer-to-peer or client-server architectures. The embodiment ofcomputing environment 100 depicted in FIG. 1 utilizes a client-serverarchitecture to, at least in part, support reference-based communicationbetween IM devices 105 and 110. Other embodiments of computingenvironment 100, however, utilize a peer-to-peer architecture in whichIM software 115 and IM device 105 and/or IM device 110 can respectivelyinclude the features and capabilities described with respect to IMauthentication software 130 and IM server 125 (e.g., embodiments inwhich the instance of IM software 115 executing on the source computingdevice provides the functionality of IM authentication software 130).

FIG. 2A is a flowchart depicting operations on a source computing devicefor supporting reference-based communication in instant messagingsystems, on a computing device within the computing environment of FIG.1, in accordance with an embodiment of the present invention. Morespecifically, FIG. 2A depicts operations 200 of an instance of IMsoftware 115 that is executing on IM device 105 (i.e., a sourcecomputing device).

In operation 202, IM software 115 captures a portion of a reference. Insome embodiments, IM software 115 includes various features that areanalogous to the features of a screenshot tool. For example, a user ofIM device 105 can take a screenshot that includes a dialogue box, aportion of a text file, a portion of a spreadsheet, or a screenshot ofvarious other objects that are presented on IM user interface 117 of IMdevice 105. In other embodiments, IM software 115 provides one or bothof the aforementioned features and the ability to copy a portion of areference. In one example, a user of IM device 105 can “left-click” on apage of a word processing document, a page of a portable document formatfile (PDF file), a page of a spreadsheet, a dialogue box, or anotherportion of an object (i.e., a reference) that is presented, at least inpart, on IM user interface 117 of IM device 105 to open a menu thatprovides an option to “copy,” and thus capture, the portion of theobject. In another example, a user of IM device 105 can use a cursor toselect a portion of an object (i.e., a reference) that is presented onIM user interface 117 of IM device 105 and similarly “copy” the selectedportion.

In operation 204, IM software 115 inserts the captured portion of thereference into an electronic message. For example, in operation 204 auser of IM device 105 can paste a screen shot of a portion of areference or paste a copied portion of a reference into a data entryfield that is presented on IM user interface 117 of IM device 105 (e.g.,a chat window of an IM application, such as IM software 115 or any fieldthat can present one or both of text and images).

In operation 206, IM software 115 associates the captured portion of thereference with a supplemental object that includes more information thatis associated with the reference than the captured portion of thereference. In some embodiments, the supplemental object is a copy of thereference. If, for example, the user of IM device 105 captures a portionof a word processing file or a spreadsheet file via IM software 115, IMsoftware 115 associates the word processing file or the spreadsheet filewith the captured portion in order to provide the functionalitydescribed herein with respect to an instance of IM software 115 that isexecuting on a target computing device (e.g., on IM device 110). Onesuch example is discussed in greater detail with respect to FIGS. 4A and4B. In other embodiments, the supplemental object is code and/or datathat, when executed by an instance of IM software 115 executing on atarget computing device, simulates, at least in part, an applicationand/or graphical user interface element (GUI element) that was captured,at least in part, via an instance of IM software 115 executing on asource computing device (e.g., on IM device 105). If, for example, theuser of IM device 105 captures a dialogue box via IM software 115, IMsoftware 115 associates the captured portion of the reference (e.g., thedialogue box as presented on the source computing device duringoperation 202) with code and data that, when executed by an instance ofIM software 115 executing on IM device 110, enables the user of IMdevice 110 to interact with the dialogue in a similar manner to the userof IM device 105. One such example is discussed in greater detail withrespect to FIGS. 3A and 3B.

In operation 208, IM software 115 stores the supplemental object that isassociated with the captured portion of the reference in a database suchthat recipients of messages including captured portions of referencescan query the database for at least portions of the supplemental objects(i.e., additional data describing the references). In some embodiments,IM software 115 sends data describing the supplemental object to IMserver 125 in operation 208. In other embodiments, IM software 115stores data describing the supplemental object to the source computingdevice (e.g., IM device 105) or, if such data is stored on the sourcecomputing device, makes such data available to authorized subject(s) inresponse to a query for at least a portion of the supplemental object.Persons of ordinary skill in the art will understand that, in suchembodiments, the data may not be available to the authorized subject(s)if the source computing device is not communicatively connected to thetarget computing device (e.g., if a user of IM device 105 “logs off”from an IM application or IM device 105 is not connected to network 120when a query is sent). In yet other embodiments, IM software 115 sendsdata describing the supplemental object to the target computing device(e.g., IM device 110) in operation 208 to enable the target computingdevice to create a local copy of the data describing the supplementalobject. In various embodiments, IM software 115 also associates datadescribing one or more access control rules and one or more authorizedsubjects with the supplemental object such that only the authorizedsubject(s) can access the data describing the supplemental object. Insome embodiments, the one or more access control rules identify arecipient of an electronic message that contains the captured portion ofthe reference (e.g., an IM or email) as an authorized subject such thata mandatory access control policy will permit the recipient to accessdata describing the supplemental object. In other embodiments, the oneor more access control rules identify a class of subjects (e.g.,technical support specialists) such that a role-based access controlpolicy will permit a recipient of the message who is also a member ofthe class to access data describing the supplemental object.

In operation 210, IM software 115 sends the electronic message thatincludes the captured portion of the reference to one or more targetcomputing device(s). In some embodiments, IM software 115 sends theelectronic message directly to computing device(s) that are associatedwith the authorized subject(s) (e.g., IM device 110 or a server thatmembers of a class of authorized subjects can access from one or morecomputing devices executing instances of IM software 115). In otherembodiments, IM software 115 sends the electronic message to theauthorized subject(s) via one or more intermediate computing nodes inaccordance with various IM protocols, email protocols, or another set ofelectronic messaging protocols known in the art.

FIG. 2B is a flowchart depicting operations on a target computing devicefor supporting reference-based communication in instant messagingsystems, on a computing device within the computing environment of FIG.1, and in accordance with an embodiment of the present invention. Morespecifically, FIG. 2B depicts operations 220 of an instance of IMsoftware 115 that is executing on IM device 110 (i.e., a targetcomputing device). Persons of ordinary skill in the art will understandthat operations 220 can also represent operations of IM software 115executing on IM device 105 in response to receiving an electronicmessage from IM device 110 (i.e., IM device 105 can be a sourcecomputing device with respect to a first message and a target computingdevice with respect to a second message).

In operation 222, an instance of IM software 115 that is executing on atarget computer device (e.g., IM device 110) receives an electronicmessage including a captured portion of a reference from a sourcecomputing device. IM user interface 117 presents the captured portion ofthe reference within the received electronic message (operation 224).

In decision 226, IM software 115 determines whether or not a request foradditional information is detected. In some embodiments, a request foradditional information can be made and detected through interactionswith the captured portion of the reference if the captured portionincludes, for example, interactive graphical elements such as buttons,text boxes, list boxes or other interactive graphical elements known inthe art. In other embodiments, a request for additional information canbe made and detected through interactions with an object of IM userinterface 117 (e.g., a toolbar button) in order to extract moreinformation from the reference than is available through the capturedportion. In yet other embodiments, a request for additional informationcan be made through either of the aforementioned techniques based on thetype of reference that is associated with the captured portion (e.g.,whether or not the reference is an interactive dialogue or a text file).

If IM software 115 does not detect a request for additional information(decision 226, NO branch), a subroutine of IM software 115 monitors IMuser interface 117 until it detects a request for additional informationor execution of IM software 115 is terminated. In various embodiments,IM software 115 permits a user of IM device 110 (i.e., a user of atarget computing device) to send and receive messages while thesubroutine monitors IM user interface 117 for requests for additionalinformation regarding captured portion(s) of one or more references.

If IM software 115 detects a request for additional information(decision 226, YES branch), IM software 115 determines if a local copyof the supplemental object exists. For example, IM software 115 candetermine that a local copy of the supplemental object exists (i) whenthe target computing device receives a copy of the supplemental objectfrom the source computing device or (ii) based on metadata indicatingthat the supplemental object is associated with data and/or code thatexists locally. If IM software 115 determines that a local copy of thesupplemental object exists (decision 228, YES branch), IM software 115extracts additional information from the supplemental object based, atleast in part, on the nature of the request, as discussed with respectto FIGS. 3A-4B (operation 230). If IM software 115 determines that alocal copy of the supplemental object does not exist (decision 228, NObranch), IM software 115, in operation 232, sends (i) a query for theadditional information and (ii) subject credentials (e.g., credentialsassociated with a user of IM device 110) to a computing device thatincludes a data repository that stores data that describes thesupplemental object (e.g., reference database 135 on IM server 125 or asource computing device, such as IM device 105, based on thearchitecture of computing environment 100). As described herein, someembodiments permit only authorized subject(s) to access the data thatdescribes the supplemental object. In such embodiments, IMauthentication software 130, for example, utilizes the credentials toauthenticate the subject and determine whether or not the subject isauthorized to receive the requested information. If the subject isauthorized to receive the requested information, or if no access controlfeatures are implemented, IM software 115 receives (e.g., from IM server125) information that was extracted from the supplemental object based,at least in part, on the query (operation 234).

In operation 236, IM software 115 modifies the electronic message thatis presented on the target computing device (e.g., a message presentedon IM user interface 117 of IM device 110) based on the additionalinformation that was extracted from the supplemental object. In someembodiments, IM software 115 modifies the electronic message by addinginteractive features of the reference to the captured portion of thereference, as discussed with respect to FIGS. 3A and 3B. In otherembodiments, IM software 115 modifies the electronic message byexpanding the captured portion of the reference such that the electronicmessage presents additional information from the reference, as discussedwith respect to FIGS. 4A and 4B.

FIGS. 3A and 3B depict an embodiment of IM user interface 117 thatenables a user of a target computer device to interact with a dialoguethat is presented in an electronic message, in accordance with anembodiment of the present invention. More specifically, FIGS. 3A and 3Bdepict the functionality of IM user interface 117 in response toreceiving, on a target computing device executing an instance of IMsoftware 115, a message including an image of a dialogue box.

In the embodiment depicted in FIG. 3A, IM user interface 117 includestoolbar 302, which includes buttons 304A and 304B for controllingvarious aspects of IM software 115. In this depiction of IM userinterface 117, “User A” is a user of IM device 105 (i.e., the sourcecomputing device) and “User B” is a user of IM device 110 (i.e., thetarget computing device). IM user interface 117 presents message 306,from User A, and message 308 from User B. Message 306 includes an imageof dialogue box 312 and indicates that User A has encountered a problemrelated to dialogue box 312 and is requesting User B's assistance. Asdepicted in FIG. 3A, dialogue box 312 includes OK button 314 and detailsbutton 316. Message 308 indicates that User B cannot assist User Awithout knowing more about the error (i.e., without information that isavailable by clicking on details button 316). IM user interface 117,however, includes status indicator 310, which indicates that User A is“away” (i.e., unavailable) and thus cannot currently respond to User B'srequest for User A to click on details button 316. Because the image ofdialogue box 312 depicted in Figure A is merely the captured portion ofthe reference (i.e., the reference is dialogue box 312 and thecorresponding, executable code) and does not have the functionality ofdialogue box 312, User B clicks on “interact with reference” button 318to, as discussed with respect to FIG. 2B, instruct IM software 115 toquery the database that stores the supplemental object and, based oninformation extracted from the supplemental object, modify message 306such that User B can click on details button 316 to view additionalinformation concerning the error. In other embodiments, IM software 115automatically queries the database to identify any interactive featuresof a captured portion of a reference and, in response to identifying oneor more interactive features, automatically modifies the capturedportion such that the captured portion of the reference includes theidentified, interactive features of the reference. In such embodiments,IM user interface 117 can omit “interact with reference button” 318.

FIG. 3B depicts IM user interface 117 of FIG. 3A after the instance ofIM software 115 executing on IM device 110 simulates a click on detailsbutton 316 of dialogue box 312 in response to User B activating“interact with reference” button 318 and clicking on details button 316.Based on the simulation, IM software 115 modifies message 306 such thatIM user interface 117 presents error details pane 320 within the imageof dialogue box 312. Even though status indicator 310 continues toindicate that User A is “away,” User B is able to advantageously obtainadditional information relating to the error from error detail pane 320without having to wait for User A to return and perform the requestedaction.

FIGS. 4A and 4B depict an embodiment of IM user interface 117 thatenables a user of a target computer device to expand a captured portionof a reference that is presented in an electronic message, in accordancewith an embodiment of the present invention. More specifically, FIGS. 4Aand 4B depict the functionality of IM user interface 117 in response toreceiving, on a target computing device executing an instance of IMsoftware 115, a message including a portion of code that was copied froma document (e.g., a text file) including additional code.

In the embodiment depicted in FIG. 4A, IM user interface 117 includestoolbar 402, which includes buttons 404A and 404B for controllingvarious aspects of IM software 115. In this depiction of IM userinterface 117, “User A” is a user of IM device 105 (i.e., the sourcecomputing device) and “User B” is a user of IM device 110 (i.e., thetarget computing device). IM user interface 117 presents message 408,from User A, and message 412, from User B. Message 408 includes aportion of code that was copied (i.e., captured) from a documentincluding additional code (i.e., the reference) and pasted, as object410, into message 408. Message 408 indicates that User A is requestingthat User B review the captured portion of code. Message 412 indicatesthat User B would like User A to include additional code to providecontext for the captured portion of the code. Status indicator 414,however, indicates that User A is “away” (i.e., unavailable) and thuscannot currently respond to User B's request for additional code. Toobtain the additional code, User B clicks on “expand reference” button406 of toolbar 402.

FIG. 4B depicts IM user interface 117 of FIG. 4A after the instance ofIM software 115 executing on IM device 110 registers a click on “expandreference” button 406 and User B utilizes IM user interface 117 toindicate the additional code that is desired. In response to registeringa click on “expand reference” button 406, the embodiment of IM userinterface 117 depicted in FIG. 4B modifies a border surrounding object410 (e.g., modifies the stroke pattern of the border) to indicate thatUser B may drag handle 416 to reveal code prior to the captured portionof code and may drag handle 418 to reveal code following the capturedportion of code. In response to IM user interface 117 registeringmovement of handle 416 or handle 418, IM software 115 queries thedatabase that stores the supplemental object (e.g., reference database135 of IM server 125) and, based on information extracted from thesupplemental object, instructs IM user interface 117 to modify object410 and message 408 in accordance with the movement of the respectivehandle and extracted information, as discussed with respect to FIG. 2B.In the embodiment, depicted in FIG. 4B, IM user interface 117 cannotconcurrently present all of the captured portion of the code and all ofthe additional code. In this embodiment, IM user interface 117 providesa horizontal scroll bar (e.g., scroll bar 420) that is associated withobject 410 to enable User B to selectively view various portions ofobject 410. In other embodiments, IM user interface 117 provides ananalogous, vertical scroll bar to enable User B to selectively viewvarious portions of object 410. In yet other embodiments, IM userinterface 117 provides both a horizontal scroll bar and a verticalscrollbar to enable User B to selectively view various portion of object410. Even though status indicator 414 continues to indicate that User Ais “away,” User B is able to advantageously obtain additional portionsof the code without having to wait for User A to return and provide therequested information.

Persons of ordinary skill in the art will understand that thefunctionality described with respect to FIGS. 4A and 4B can be providedfor various other documents, such as other types of word processingfiles, various types of spreadsheet files, PDF files, and various othertypes of documents. In other embodiments, for example, object 410 is acaptured portion of a spreadsheet, and upon registering a click on“expand reference” button 406, IM user interface 117 provides a firstpair of handles and a second pair of handles to respectively exposeadditional rows and additional columns of the spreadsheet. In suchembodiments, IM user interface 117 can provide a horizontal scroll barand a vertical scroll bar that are associated with object 410 to enableUser B to selectively view various columns and rows of the capturedportion of the spread sheet and the expanded portion of the spreadsheet.

In addition persons of ordinary skill in the art will understand thatthe functionality described with respect to IM software 115, IM userinterface 117, and IM server 125 are not limited to IM computingenvironments, such as computing environment 100. For example, emailapplications can advantageously include similar functionality in orderto enable recipients of emails that include captured portions ofreferences to obtain additional information relating to such referenceswithout requiring the participation of the user who sent the email thatincludes the captured portion of the reference.

FIG. 5 is a block diagram of components of a computing device, generallydesignated 500, in accordance with an embodiment of the presentinvention. In one embodiment, computing system 500 is representative ofone or more of IM device 105, IM device 110, and IM server 125 withincomputing environment 100, in which case each of IM device 105 and IMdevice 110 includes IM software 115 and IM server 125 includes IMauthentication software 130.

It should be appreciated that FIG. 5 provides only an illustration ofone implementation and does not imply any limitations with regard to theenvironments in which different embodiments may be implemented. Manymodifications to the depicted environment may be made.

Computing system 500 includes processor(s) 504, cache 516, memory 506,persistent storage 508, input/output (I/O) interface(s) 512,communications unit 510, and communications fabric 502. Communicationsfabric 502 provides communications between cache 516, memory 506,persistent storage 508, communications unit 510, and input/output (I/O)interface(s) 512. Communications fabric 502 can be implemented with anyarchitecture designed for passing data and/or control informationbetween processors (such as microprocessors, communications and networkprocessors, etc.), system memory, peripheral devices, and any otherhardware components within a system. For example, communications fabric502 can be implemented with one or more buses or a crossbar switch.

Memory 506 and persistent storage 508 are computer readable storagemedia. In this embodiment, memory 506 includes random access memory(RAM). In general, memory 506 can include any suitable volatile ornon-volatile computer readable storage media. Cache 516 is a fast memorythat enhances the performance of processor(s) 504 by holding recentlyaccessed data, and data near recently accessed data, from memory 506.

Program instructions and data used to practice embodiments of thepresent invention may be stored in persistent storage 508 and in memory506 for execution by one or more of the respective processor(s) 504 viacache 516. In an embodiment, persistent storage 508 includes a magnetichard disk drive. Alternatively, or in addition to a magnetic hard diskdrive, persistent storage 508 can include a solid state hard drive, asemiconductor storage device, read-only memory (ROM), erasableprogrammable read-only memory (EPROM), flash memory, or any othercomputer readable storage media that is capable of storing programinstructions or digital information.

The media used by persistent storage 508 may also be removable. Forexample, a removable hard drive may be used for persistent storage 508.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer readable storage medium that is also part of persistent storage508.

Communications unit 510, in these examples, provides for communicationswith other data processing systems or devices. In these examples,communications unit 510 includes one or more network interface cards.Communications unit 510 may provide communications through the use ofeither or both physical and wireless communications links. Programinstructions and data used to practice embodiments of the presentinvention may be downloaded to persistent storage 508 throughcommunications unit 510.

I/O interface(s) 512 allows for input and output of data with otherdevices that may be connected to computer system 500. For example, I/Ointerface(s) 512 may provide a connection to external device(s) 518 suchas a keyboard, keypad, a touch screen, and/or some other suitable inputdevice. External device(s) 518 can also include portable computerreadable storage media such as, for example, thumb drives, portableoptical or magnetic disks, and memory cards. Software and data used topractice embodiments of the present invention can be stored on suchportable computer readable storage media and can be loaded ontopersistent storage 508 via I/O interface(s) 512. I/O interface(s) 512also connect to display 520.

Display 520 provides a mechanism to display or present data to a userand may be, for example, a computer monitor.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

As used herein, a list of alternatives such as “at least one of A, B,and C” should be interpreted to mean “at least one A, at least one B, atleast one C, or any combination of A, B, and C.”

Additionally, the phrase “based on” should be interpreted to mean“based, at least in part, on.”

The term “exemplary” means of or relating to an example and should notbe construed to indicate that any particular embodiment is preferredrelative to any other embodiment.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

1. A method for supporting reference-based communication in instantmessaging systems, comprising: a recipient receiving, by one or morecomputer processors, an electronic message that includes a capturedportion of a reference, wherein the captured portion of the reference isan object representing, at least in part, a visual portion of thereference that is selected and inserted into the electronic message by asender; presenting within the electronic message, by one or morecomputer processors, a visual representation of the captured portion ofthe reference on a user interface; sending to a database that stores asupplemental object, by one or more computer processors, a query fordata that is associated with the reference, and in response, receiving,by one or more computer processors, information extracted from thesupplemental object based on the query, wherein the supplemental objectrepresents a greater visual portion of the reference than the capturedportion of the reference; and on the user interface, modifying, by oneor more computer processors, the visual representation of the capturedportion of the reference within the electronic message without anadditional action from the sender and based on (i) the informationextracted from the supplemental object and (ii) an interaction with thecaptured portion of the reference that generated the query, wherein theinteraction represents a request to extend the captured portion of thereference in at least one direction on the user interface and therebypresent a greater visual portion of the reference than the capturedportion of the reference. 2-4. (canceled)
 5. The method of claim 1,further comprising: detecting, by one or more computer processors, analteration of the captured portion of the reference via the userinterface, wherein the alteration is a request to expand the visualrepresentation of the captured portion of the reference; and generating,by one or more computer processors, the query based on the alteration ofthe captured portion of the reference.
 6. The method of claim 5,wherein: the reference is a text document; the captured portion of thereference is one or more lines of text from the text document; thedatabase stores a copy of the text document as the supplemental object;the extracted information describes one or more additional lines oftext; and modifying the electronic message based, at least in part, onthe extracted information and an interaction with captured portion ofthe reference adds the one or more additional lines of text to thecaptured portion of the reference.
 7. The method of claim 1, furthercomprising: sending, by one or more computer processors, credentialsthat are associated with a recipient of the electronic message to acomputing device that manages the database, wherein the credentialsindicate that the recipient is authorized to access the supplementalobject stored in the database.
 8. A computer program product forsupporting reference-based communication in instant messaging systems,the computer program product comprising: a computer readable storagedevice and program instructions stored on the computer readable storagedevice, the program instructions comprising: program instructions toreceive, at a recipient, an electronic message that includes a capturedportion of a reference, wherein the captured portion of the reference isan object representing, at least in part, a visual portion of thereference that is selected and inserted into the electronic message by asender; program instructions to present, within the electronic message,a visual representation of the captured portion of the reference on auser interface; program instructions to send, to a database that storesa supplemental object, a query for data that is associated with thereference, and in response, receiving, by one or more computerprocessors, information extracted from the supplemental object based onthe query, wherein the supplemental object represents a greater visualportion of the reference than the captured portion of the reference; andprogram instructions to, on the user interface, modify the visualrepresentation of the captured portion of the reference within theelectronic message without an additional action from the sender andbased on (i) the information extracted from the supplemental object and(ii) an interaction with the captured portion of the reference thatgenerated the query, wherein the interaction represents a request toextend the captured portion of the reference in at least one directionon the user interface and thereby present a greater visual portion ofthe reference than the captured portion of the reference. 9-11.(canceled)
 12. The computer program product of claim 8, the programinstructions further comprising: program instructions to detect analteration of the captured portion of the reference via the userinterface, wherein the alteration is a request to expand the visualrepresentation of the captured portion of the reference; and programinstructions to generate the query based on the alteration of thecaptured portion of the reference.
 13. The computer program product ofclaim 12, wherein: the reference is a text document; the capturedportion of the reference is one or more lines of text from the textdocument; the database stores a copy of the text document as thesupplemental object; the extracted information describes one or moreadditional lines of text; and the program instructions to modify theelectronic message based, at least in part, on the extracted informationand an interaction with captured portion of the reference add the one ormore additional lines of text to the captured portion of the reference.14. The computer program product of claim 8, the program instructionsfurther comprising: program instructions to send credentials that areassociated with a recipient of the electronic message to a computingdevice that manages the database, wherein the credentials indicate thatthe recipient is authorized to access the supplemental object stored inthe database.
 15. A computer system for supporting reference-basedcommunication in instant messaging systems, the computer systemcomprising: one or more computer processors; one or more computerreadable storage devices; program instructions stored on the one or morecomputer readable storage devices for execution by at least one of theone or more processors, the program instructions comprising: programinstructions to receive, at a recipient, an electronic message thatincludes a captured portion of a reference, wherein the captured portionof the reference is an object representing, at least in part, a visualportion of the reference that is selected and inserted into theelectronic message by a sender; program instructions to present, withinthe electronic message, a visual representation of the captured portionof the reference on a user interface; program instructions to send, to adatabase that stores a supplemental object, a query for data that isassociated with the reference, and in response, receiving, by one ormore computer processors, information extracted from the supplementalobject based on the query, wherein the supplemental object represents agreater visual portion of the reference than the captured portion of thereference; and program instructions to, on the user interface, modifythe visual representation of the captured portion of the referencewithin the electronic message without an additional action from thesender and based on (i) the information extracted from the supplementalobject and (ii) an interaction with the captured portion of thereference that generated the query, wherein the interaction represents arequest to extend the captured portion of the reference in at least onedirection on the user interface and thereby present a greater visualportion of the reference than the captured portion of the reference.16-18. (canceled)
 19. The computer system of claim 15, the programinstructions further comprising: program instructions to detect analteration of the captured portion of the reference via the userinterface, wherein the alteration is a request to expand the visualrepresentation of the captured portion of the reference; and programinstructions to generate the query based on the alteration of thecaptured portion of the reference.
 20. The computer system of claim 19,wherein: the reference is a text document; the captured portion of thereference is one or more lines of text from the text document; thedatabase stores a copy of the text document as the supplemental object;the extracted information describes one or more additional lines oftext; and the program instructions to modify the electronic messagebased, at least in part, on the extracted information and an interactionwith captured portion of the reference add the one or more additionallines of text to the captured portion of the reference.
 21. The computersystem of claim 15, the program instructions further comprising: programinstructions to send credentials that are associated with a recipient ofthe electronic message to a computing device that manages the database,wherein the credentials indicate that the recipient is authorized toaccess supplemental object stored in the database.
 22. The method ofclaim 1, wherein the captured portion of the reference is a screenshotof a portion of the reference that was taken by the sender.
 23. Themethod of claim 1, wherein the captured portion of the reference is acopied portion of the reference that was copied by the sender.
 24. Thecomputer program product of claim 8, wherein the captured portion of thereference is a screenshot of a portion of the reference that was takenby the sender.
 25. The computer program product of claim 8, wherein thecaptured portion of the reference is a copied portion of the referencethat was copied by the sender.
 26. The computer system of claim 15,wherein the captured portion of the reference is a screenshot of aportion of the reference that was taken by the sender.
 27. The computersystem of claim 15, wherein the captured portion of the reference is acopied portion of the reference that was copied by the sender.